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1  PROTOCOL  FOR  EQUIPMENT  VERIFICATION  TESTING 

2  OF  DISINFECTION  BY-PRODUCT  PRECURSOR  REMOVAL 

3  BY  PACKAGED  AND/OR  MODULAR 

4  DRINKING  WATER  TREATMENT  SYSTEMS 

5 

6  1.0  INTRODUCTION 

7 

8  This  document  is  the  study  protocol  to  be  used  for  verification  testing  of  equipment  designed  to 

9  achieve  removal  of  precursors  to  disinfection  by-products  (DBFs).  In  order  to  participate  in  the 

1 0  equipment  verification  process,  the  equipment  Manufacturer  must  adhere  to  the  requirements  of  this 

11  study  protocol  in  developing  a  Manufacturer  Field  Operations  Document  (FOD).  The  final 

12  submission  of  the  Manufacturer  FOD  shall: 

13 

14  •  include  the  information  requested  in  this  protocol; 

15  •  conform  to  the  format  identified  herein;  and 

16  •  conform  to  the  specific  NSF  International  (NSF)  Equipment  Verification  Testing  Plan  or 

17  Plans  related  to  the  statement  or  statements  of  capabilities  that  are  to  be  verified. 

18 

19  The  Manufacturer  FOD  may  include  more  than  one  Testing  Plan.  Equipment  testing  might  be 

20  undertaken  to  verify  performance  of  a  packaged  plant  systems  employing  processes  that  may  include 

21  but  are  not  limited  to  coagulation/clarification,  oxidation  or  mixed  oxidation  processes,  adsorption, 

22  biological  filtration  or  membrane  filtration  for  removal  of  DBP  precursors. 

23 

24  This  protocol  document  is  presented  in  two  fonts.  The  non-italicized  font  provides  background 

25  information  that  the  Manufacturer  may  find  useful  in  preparation  of  the  Manufacturer  FOD.  The 

26  italicized  text  indicates  specific  study  protocol  deliverables  that  are  required  of  the  Manufacturer 

27  and  that  must  be  incorporated  in  the  Manufacturer  FOD. 

28 

29  The  following  glossary  terms  are  presented  here  for  subsequent  reference  in  this  protocol: 

30 

31  •  Certification  -  the  attestation  that  a  piece  of  equipment  and/or  a  device  has  met  all 

32  applicable  requirements,  e.g.,  standard  performance  criteria  and  policies,  and  continues  to 

33  meet  all  applicable  requirements. 

34  •  Distribution  System  -  a  system  of  conduits  by  which  a  primary  water  supply  is  conveyed  to 

35  consumers,  typically  by  a  network  of  pipelines. 

36  •  Manufacturer  -  a  business  that  assembles  and/or  sells  package  plant  equipment  and/or 

37  modular  systems.  The  role  of  the  Manufacturer  is  to  provide  the  package  plant  and/or 

38  modular  system  and  technical  support  during  the  Verification  Testing  Program.  The 

39  Manufacturer  is  also  responsible  for  providing  assistance  to  the  third  party  testing 
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1  organization  during  operation  and  monitoring  of  the  package  plant  or  modular  system  in 

2  the  Verification  Testing  Program. 

3  •  Manufacturer  Field  Operations  Document  (FOD)  -  a  document  of  field  testing  operations 

4  and  procedures.  The  document  will  be  prepared  by  Manufacturer  or  by  third  party  on 

5  behalf  of  Manufacturer  and  will  include  the  specific  details  of  the  experimental  approach  in 

6  the  section  titled  Field  Operations  Procedures. 

7  •  Modular  Sv.stem  -  A  functional  assembly  of  components  for  use  in  a  drinking  water 

8  treatment  system  or  packaged  plant,  each  part  of  which  provides  a  limited  form  of 

9  treatment  of  the  feed  waterfsk  Treated  waters  mav  be  discharged  to  another  packaged 

10  plant  module  or  to  the  distribution  system  if  the  modular  system  includes  the  final  step  of 

11  treatment. 

12  •  NSF  Equipment  Verification  Testing  Plan  -  a  specific  testing  plan  for  each  packaged  plant 

13  technology  application,  such  as  systems  employing  coagulation/clarification,  oxidation  or 

14  mixed  oxidation  processes,  adsorption,  biological  filtration  or  membrane  filtration,  and 

15  other  processes  for  removal  of  DBP  precursors.  This  plan  will  be  developed  by  NSF  for 

16  the  Manufacturer  to  assist  in  development  of  the  Manufacturer  FOD  for  the  Verification 

17  Testing  Program. 

18  •  Plant  Operator  -  the  person  working  for  a  small  water  system  who  is  responsible  for 

19  operating  packaged  water  treatment  equipment  to  produce  treated  drinking  water.  This 

20  person  may  also  collect  samples,  record  data  and  attend  to  the  daily  operations  of 

21  equipment  throughout  the  testing  periods. 

22  •  Packaged  plant  -  a  complete  water  treatment  system  including  all  components  from 

23  connection  to  the  feed  water(s)  through  discharge  to  the  distribution  system. 

24  •  Study  Protocol  for  Equipment  Verification  Testing  -  this  document.  Protocol  will  be  used 

25  for  reference  during  Manufacturer  participation  in  Verification  Testing  Program. 

26  •  Testing  Organization  -  an  organization  qualified  to  perform  studies  and  testing  of  package 

27  or  modular  systems.  The  role  of  the  testing  organization  is  to  ensure  that  there  is  skilled 

28  operation  of  a  package  plant  during  the  intense  periods  of  testing  and  that  all  of  the  tasks 

29  required  by  the  Study  Protocol  for  Equipment  Verification  Testing  are  performed 

30  properly.  The  Testing  Organization  is  responsible  for: 

31  managing,  evaluating,  interpreting  and  reporting  on  the  data  produced  by  the 

32  verification  testing  and  study; 

33  =>  providing  logistical  support,  scheduling  and  coordinating  the  activities  of  all 

34  participants  in  the  verification  testing  and  study,  i.e.,  establishing  a  communieations 

35  network; 

36  =*^  advising  the  Manufacturer  on  feed  water  quality  and  test  site  seleetion.  such  that 

37  the  locations  selected  for  the  verification  testing  and  study  have  feed  water  quality 

38  consistent  with  the  objectives  of  the  Study  Protocol  for  Equipment  Verification  Testing. 
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•  Verification  -  to  establish  the  evidence  on  the  range  of  performance  of  equipment  and/or 
device  under  specific  conditions  following  a  predetermined  study  protocol. 

•  Water  System  -  the  water  system  that  operates  using  packaged  water  treatment  equipment 
to  provide  treated  water  to  its  customers. 

1.1  Background 

U.S.  Environmental  Protection  Agency  (EPA)  has  partnered  with  NSF,  a  nonprofit  testing  and 
certification  organization,  to  verify  performance  of  small  packaged  drinking  water  systems  that  serve 
small  communities.  It  is  expected  Aat  both  the  domestic  and  international  markets  for  such  systems 
are  substantial.  EPA  and  NSF  have  formed  an  oversight  stakeholders  group  composed  of  buyers, 
sellers,  and  state  permitters,  to  assist  in  formulating  consensus  testing  protocols.  A  goal  of 
verification  testing  is  to  enhance  and  facilitate  the  acceptance  of  small  packaged  drinking  water 
treatment  equipment  by  state  drinking  water  regulatory  engineers  and  consulting  engineers  whife 
reducing  the  need  for  testing  of  equipment  at  each  location  where  the  equipment  use  is  contemplated. 

NSF  will  meet  this  goal  by  working  with  equipment  Manufacturers  and  other  agencies  in  planning 
and  conducting  Equipment  Verification  Testing  Programs,  evaluating  data  generated  by  such  testing 
and  managing  and  disseminating  information.  The  Manufacturer  is  expected  to  secure  the  appropriate 
resources  to  support  their  part  of  the  equipment  verification  process,  including  provision  of 
equipment  and  technical  support. 

The  verification  process  established  by  EPA  and  NSF  is  intended  to  serve  as  a  template  for 
conducting  water  treatment  verification  tests  that  will  generate  high  quality  data  for  verification  of 
equipment  performance.  The  verification  process  is  a  model  process  that  can  help  in  moving  small 
packaged  and/or  modular  drinking  water  treatment  equipment  into  routine  use  more  quickly.  The 
verification  of  an  equipment's  performance  involves  five  sequential  steps  : 

1.  Development  of  a  Verification/Manufacturer  FOD; 

2.  Execution  of  verification  testing; 

3.  Data  reduction,  analysis,  and  reporting; 

4.  Performance  and  cost  (labor,  chemicals,  energy)  verification; 

5.  Report  preparation  and  information  transfer. 

1.2  Objectives 

The  specific  objectives  of  the  Equipment  Verification  Testing  Program  may  be  different  for  each 
Manufacturer,  depending  upon  the  statement  of  capabilities  of  the  specific  equipment  to  be  tested. 
The  objectives  developed  by  each  Manufacturer  will  be  defined  and  described  in  detail  in  the 
Manufacturer  FOD  developed  for  each  piece  of  equipment.  The  objectives  of  the  Equipment 
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1  Verification  Testing  Program  may  include: 

2 

3  •  Generation  of  field  data  appropriate  for  verifying  the  performance  of  the  equipment; 

4  •  Generation  of  field  data  in  support  of  meeting  current  or  anticipated  water  quality 

5  regulations; 

6  •  Evaluation  of  new  advances  in  equipment  and  equipment  design. 

7 

8  An  important  aspect  in  the  development  of  verification  testing  is  to  describe  the  procedures  that  will 

9  be  used  to  verify  the  statement  of  performance  capabilities  made  for  water  treatment  equipment.  A 

10  verification  testing  plan  document  shall  incorporate  the  QA/QC  elements  needed  to  provide  data  of 

11  appropriate  quality  sufficient  to  reach  a  defensible  position  regarding  the  equipment  performance. 

12  Verification  testing  conducted  at  a  single  site  may  not  represent  every  environmental  situation  which 

13  may  be  acceptable  for  the  equipment  tested,  but  it  will  provide  data  of  sufficient  quality  to  make  a 

1 4  judgment  about  the  application  of  the  equipment  under  conditions  similar  to  those  encountered  in  the 

15  verification  testing. 

16 

17  It  is  important  to  note  that  verification  of  the  equipment  does  not  mean  that  the  equipment  is 

18  “certified”  by  NSF  or  EPA.  Rather,  it  recognizes  that  the  performance  of  the  equipment  has  been 

19  determined  and  verified  by  these  organizations. 

20 
21 

22  1.3  Scope 

23 

24  This  protocol  outlines  the  verifieation  process  for  equipment  designed  to  achieve  removal  of 

25  precursors  to  DBFs.  The  scope  of  this  protoeol  includes  testing  plans  for  packaged  and/or  modular 

26  drinking  water  treatment  systems  designed  to  achieve  removal  of  DBP  precursors.  This  protocol  is 

27  not  an  NSF  or  third-party  consensus  standard  and  it  does  not  endorse  the  packaged  plants  or 

28  technologies  described  herein. 

29 

30  An  overview  of  the  equipment  verification  process  and  the  elements  of  the  Manufacturer  FOD  to  be 

31  developed  by  the  Manufacturer  are  described  in  this  protocol  document.  Specifically,  the 

32  Manufacturer  FOD  shall  define  the  following  elements  of  the  verification  testing: 

33 

34  •  Roles  and  responsibilities  of  verification  testing  participants; 

35  •  Procedures  governing  verification  testing  activities  such  as  equipment  operation  and 

36  process  monitoring;  sample  collection,  preservation,  and  analysis;  and  data  collection  and 

37  interpretation; 

38  •  Experimental  design  of  the  Field  Operations  Procedures; 

39  •  Quality  assurance  (QA)  and  quality  control  (QC)  procedures  for  conducting  the 
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1  verification  testing  and  for  assessing  the  quality  of  the  data  generated  from  the  verification 

2  testing;  and, 

3  •  Health  and  safety  measures  relating  to  biohazard,  electrical,  mechanical  and  other  safety 

4  codes. 

5 

6  Content  of  Manufacturer  Field  Operations  Document; 

7 

8  The  structure  of  the  Manufacturer  FOD  must  conform  to  the  outline  below:  The  requird 

9  components  of  the  Document  will  be  described  in  greater  detail  in  the  sections  below. 

10 

11  •  TITLE  PAGE 

12  •  FOREWORD 

13  •  TABLE  OF  CONTENTS  -  The  Table  of  Contents  for  the  Manufacturer  FOD  should 

14  include  the  headings  provided  in  this  document  although  they  may  be  modified  as 

15  appropriate  for  a  particular  type  of  equipment  to  be  tested. 

16  •  EXECUTIVE  SUMMARY  -  The  Executive  Summary  describes  the  contents  of  the 

17  Manufacturer  FOD  (not  to  exceed  two  pages).  A  general  description  of  the  equipment 

18  and  the  statement  of  performance  capabilities  which  will  be  verified  during  testing  shall 

19  be  included,  as  well  as  the  testing  locations,  a  schedule,  and  a  list  of  participants. 

20  •  ABBREVIATIONS  AND  ACRONYMS  -  A  list  of  the  abbreviations  and  acronyms  used  in 

21  the  Manufacturer  Field  Operations  Document  should  be  provided. 

22  •  EQUIPMENT  VERIFICA TION  TESTING  RESPONSIBILITIES  (described  in  the  sections 

23  below) 

24  •  EQUIPMENT  CAPABILITIES  AND  DESCRIPTION  (described  in  the  sections  below) 

25  •  EXPERIMENTAL  DESIGN  (described  in  the  sections  below) 

26  •  FIELD  OPERATIONS  PROCEDURES  (described  in  the  section  below) 

27  •  QUALITY  ASSURANCE  TESTING  PLAN  (described  in  the  section  below) 

28  •  DATA  MANAGEMENT  AND  ANALYSIS  (described  in  the  section  below) 

29  •  SAFETY  PLAN  (described  in  the  section  below) 

30 

31  Manufacturer  Responsibilities: 

32 

33  Preparation  of  a  Manufacturer  Field  Operations  Bcument  that  includes  the  information  requested 

34  and  conforms  to  the  requirements  stipulated  in  this  protocol  document,  and  the  applicable  NE 

35  Equipment  Verification  Testing  Plan  or  Plans. 

36 


This  Draft  Protocol  shall  not  be  duplicated  in  whole  or  in  part  without  the  written  permission  of  NSF  International 


Page  6 


1  2.0  EQUIPMENT  VERIFICATION  TESTING  RESPONSIBILITIES 

2 

3  2.1  Verification  Testing  Organization  and  Participants 

4 

5  This  Verification  Testing  Program  is  being  conducted  by  NSF  International  with  participation  of 

6  manufacturers,  under  the  sponsorship  of  the  EPA  Office  of  Research  and  Development,  National  Risk 

7  Management  Research  Laboratory,  Water  Supply  and  Water  Resources  Division  (WSWRD)  - 

8  Cincinnati,  Ohio.  The  WSWRD  and  NSF  jointly  are  administering  the  Equipment  Verification  Testing 

9  Program.  The  NSF's  role  is  to  provide  technical  and  administrative  leadership  and  support  in 

10  conducting  the  testing. 

11 

12  The  specific  responsibilities  of  each  participant  are  discussed  in  Section  2.4.  The  required  content 

13  of  the  Manufacturer  FOD  and  the  Manufacturer  Responsibilities  are  listed  at  the  end  of  each  section. 

14  In  the  development  of  a  Manufacturer  FOD,  Manufacturers  shall  provide  a  table  including  the  name, 

15  affiliation,  and  mailing  address  of  each  participant,  a  point  of  contact,  description  of  participant’s  role, 

16  telephone  and  fax  numbers,  and  e-mail  address. 

17 

18  2.2  Verification  Testing  Agreement 

19 

20  After  equipment  has  been  accepted  by  NSF  into  die  Environmental  Technology  Verification  Program, 

21  a  letter  agreement  will  be  signed  between  the  Manufacturer  and  the  NSF.  The  purpose  of  the 

22  agreement  is  to  specify  a  fi-amework  of  responsibilities  for  conducting  the  Equipment  Verification 

23  Testing  Program. 

24 

25  It  is  important  to  note  that  the  entire  Manufacturer  FOD,  including  a  Quality  Assurance  Project  Plan 

26  (QAPP),  must  be  approved  by  the  Manufacturer  and  the  NSF  before  the  verification  testing  can 

27  proceed. 

28 

29  2.3  Organization 

30 

31  The  organizational  structure  for  the  verification  testing  showing  lines  of  communication  shall  be 

32  provided  by  the  Manufacturer. 

33 

34  2.4  Verification  Testing  Site  Name  and  Location 

35 

36  This  section  discusses  background  information  on  the  verification  testing  site(s),  with  emphasis  on 

37  the  quality  of  the  feed  water,  which  in  some  cases  may  be  the  source  water  at  the  site.  The 

38  Manufacturer  FOD  must  provide  the  site  names  and  locations  at  which  the  equipment  will  be  tested. 

39  In  most  cases,  the  equipment  will  be  demonstrated  at  more  than  one  site.  In  all  cases  the  equipment 

4  0  should  be  tested  under  different  conditions  of  feed  water  quality  (or  source  water  quality)  and  a  range 

41  of  seasonal  climate  and  weather  conditions. 
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1  2.5  Site  Characteristics 

2 

3  The  Manufacturer  FOD  must  include  a  description  of  the  test  site.  This  shall  include  a  description  of 

4  where  the  equipment  will  be  located.  If  the  feed  water  to  the  packaged  plant  equipment  is  the  source 

5  water  for  an  existing  water  treatment  plant,  describe  the  raw  water  intake,  the  opportunity  to  obtain 

6  raw  water  without  the  addition  of  any  chemicals,  and  the  operational  pattern  of  raw  water  pumping 

7  at  the  full-scale  facility  (is  it  continuous  or  intermittent?).  The  source  water  characteristics  shall  be 

8  described  and  documented.  The  Manufacturer  FOD  shall  also  describe  facilities  to  be  used  for 

9  handling  the  treated  water  and  wastes  (i.e.,  residuals)  produced  during  the  Verification  Testing 

10  Program.  Can  the  required  water  flows  and  waste  flows  produced  be  dealt  with  in  an  acceptable 

11  way?  Are  water  pollution  discharge  permits  needed? 

12 

13  2.6  Responsibilities 

14 

15  This  section  identifies  the  organizations  involved  in  the  testing  and  describes  the  primary 

16  responsibilities  of  each  organization.  The  responsibilities  of  the  Manufacturer  will  vary  depending 

17  on  the  type  of  verification  testing.  Multiple  Manufacturer  testing  for  removal  of  DBP  precursors  may 

18  be  conducted  concurrently,  and  be  fully  in  compliance  with  the  NSF  Equipment  Verification  Testing 

19  Program. 

20 

21  NSF  and  the  equipment  testing  organization  shall  be  responsible  for: 

22 

23  •  Providing  needed  logistical  support,  establishing  a  communication  network,  and 

24  scheduling  and  coordinating  the  activities  of  all  verification  testing  participants; 

25  •  Advising  the  Manufacturer  on  feed  water  quality  and  test  site  selection,  such  that  the 

26  locations  selected  as  test  sites  have  feed  water  quality  consistent  with  die  objectives  of  the 

27  verification  testing  (Manufacturer  may  recommend  a  verification  testing  site(s)); 

28  •  Managing,  evaluating,  interpreting,  and  reporting  on  data  generated  by  the  verification 

29  testing; 

30  •  Evaluating  and  reporting  on  the  performance  of  the  DBP  precursor  removal  technologies. 

31 

32  The  Manufacturer  shall  be  responsible  for  provision  of  the  equipment  to  be  evaluated.  See  additional 

33  Manufacturer  responsibilities  listed  below. 

34 

35 

36  Content  of  Manufacturer  Field  Operations  Document  Regarding  Equipment  Verification 

37  Testing  Responsibilities: 

38 

39  The  Manufacturer,  in  consultation  with  NSF  as  the  technical  lea/ishall  be  responsible  for  including 
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the  following  elements  in  the  Manufacturer  Field  Operations  Document: 

•  Definition  of  the  roles  and  responsibilities  of  appropriate  verification  testing  participants 

•  A  table  which  includes  the  name,  affiliation,  and  mailing  address  of  each  participant,  a 
point  of  contact,  description  of participant's  role,  telephone  and  fax  numbers,  and  e-mail 
address, 

•  Organization  of  operational  and  analytical  support 

•  List  of  the  site  name(s)  and  location(s). 

•  Description  of  the  test  site(s),  the  site  characteristics  and  identification  of  where  the 
equipment  will  be  located. 

Manufacturer  Responsibilities: 

•  Provision  of  complete,  field-ready  equipment  for  verification  testing; 

•  Provision  of  logistical,  and  technical  support,  as  required. 

•  Provision  of  technical  assistance  to  the  qualified  testing  organization  during  operation 
and  monitoring  of  the  equipment  undergoing  verification  testing. 

3.0  EQUIPMENT  CAPABILITIES  AND  DESCRIPTION 

3.1  Equipment  Capabilities 


The  Manufacturer  shall  identify  the  water  quality  objectives  to  be  achieved  in  the  statement  of 
performance  capabilities  of  the  equipment  to  be  evaluated  in  the  verification  testing.  Statements 
should  also  be  made  regarding  the  applications  of  the  equipment,  the  known  limitations  of  the 
equipment  and  what  advantages  it  provides  over  existing  equipment.  The  statement  of 
performance  capabilities  must  be  specific  and  verifiable  by  a  statistical  analysis  of  the  data.  Two 
examples  of  satisfactory  statements  of  performance  capabilities  are  provided  below: 


1 .  ’This  packaged  plant  is  capable  of  achieving  40%  removal  of  dissolved  organic  carbon 
(DOC)  in  feed  waters  with  total  organic  carbon  concentrations  between  2.0  and  4.0  mg/L  and 
with  feed  water  alkalinities  less  than  60  mg/L  as  CaC03." 

2.  "This  packaged  plant  is  capable  of  achieving  40%  removal  of  precursors  to  trichloroacetic 
acid  (TCA)  in  feed  waters.  Removal  of  TCA  precursors  will  be  quantified  by  comparison  of  SDS 
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1  testing  results  generated  for  feed  and  finished  water  samples.  The  following  equation  shall  be 

2  used  to  determine  percent  removal  of  all  DBP  precursors:" 

3 

Feedwater  DBP  Concentration  -  Finished  water  DBP  Concentration 
%  Removal  Precursor  Material  100  ,  ^  . 

y.  Feedwater  DBP  Concentration 


5  A  statement  of  performance  capabilities  such  as:  "This  packaged  plant  will  achieve  removal  of 

6  DOC  in  accordance  with  the  Enhanced  Coagulation  requirement  of  the  Disinfectants/Disinfection 

7  By-Product  Rule  (D/DBP  Rule)  on  a  consistent  and  dependable  basis,"  would  not  be  acceptable. 

8 

9  The  Manufacturer  shall  be  responsible  for  identification  of  which  DBP  precursors  shall  be 

10  monitored  for  removal  under  the  statement  of  performance  capabilities.  The  statement  of 

11  performance  capabilities  prepared  bv  the  Manufacturer  shall  also  indicate  the  range  of  water 

12  quality  under  which  the  equipment  can  be  challenged  while  successfully  treating  the  feed  water. 

13  Statements  of  performance  capabilities  that  are  too  easily  met  may  not  be  of  interest  to  the 

14  potential  user,  while  performance  capabilities  that  are  overstated  may  not  be  achievable.  The 

15  statement  of  performance  capabilities  forms  the  basis  of  the  entire  Equipment  Verification  Testing 

16  Program  and  must  be  chosen  appropriately.  Therefore,  the  design  of  the  Manufacturer  FOD 

17  should  include  a  sufficient  range  of  feed  water  quality  to  permit  verification  of  the  statement  of 

18  performance  capabilities. 

19 

20  It  should  be  noted  that  many  of  the  packaged  and/or  modular  drinking  water  treatment  systems 

21  participating  in  the  DBP  Precursor  Removal  Verification  Testing  Program  will  be  capable  of 

22  achieving  multiple  water  treatment  objectives.  Although  this  DBP  Precursor  Protocol  and  the 

23  associated  Verification  Testing  Plans  are  oriented  towards  removal  of  DBP  precursors,  the 

24  Manufacturer  may  want  to  look  at  the  treatment  system’s  removal  capabilities  for  additional  water 

25  quality  parameters. 

26 

27  3.2  Equipment  Description 

28 

29  Description  of  the  equipment  for  the  Verification  Testing  Program  shall  be  provided  by  the 

30  Manufacturer.  Data  plates  shall  be  permanent  and  securely  attached  to  each  production  unit.  The 

31  data  plate  shall  be  easy  to  read  in  English  or  the  language  of  the  intended  user,  located  on  the 

32  equipment  where  it  is  readily  accessible,  and  contain  at  least  the  following  information: 

33  a.  Equipment  Name 

34  b.  Model  # 
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1  c.  Manufacturer’s  name  and  address 

2  d.  Electrical  requirements  -  volts,  amps,  and  Hertz 

3  e.  Serial  Number 

4  f.  Warning  and  Caution  statements  in  legible  and  easily  discernible  print  size 

5  g.  Capacity  or  output  rate  (if  applicable) 

6 

7  Content  of  Manufacturer  Field  Operations  Document  Regarding  Equipment  Capabilities 

8  and  Description: 

9 

10  The  Manufacturer  shall  he  responsible  for  including  the  following  elements  in  the  Manufacturer 

11  Field  Operations  Document: 

12 

13  •  Description  of  the  equipment  to  be  demonstrated; 

14  •  Brief  introduction  and  discussion  of  the  engineering  and  scientific  concepts  on  which  the 

15  DBF  precursor  removal  capabilities  of  the  water  treatment  equipment  are  based; 

16  •  Description  of  the  packaged  treatment  plant  and  each  process  included  as  a  component 

17  in  the  modular  system; 

18  •  Brief  description  of  the  physical  construction/components  of  the  equipment.  Include 

19  general  environmental  requirements  and  limitations,  required  consumables;  weight, 

20  transportability,  ruggedness,  power  and  other  needed,  etc. 

21  •  Statement  of  typical  rates  of  consumption  ofchemicals.  a  description  of  the  physical  and 

22  chemical  nature  of  wastes,  and  rates  of  waste  (concentrates,  residues,  etc.); 

23  •  Definition  of  the  performance  range  of  the  equipment; 

24  •  Identification  of  any  special  licensing  requirements  associated  with  the  operation  of  the 

25  equipment; 

26  •  Description  of  the  applications  of  the  equipment  andthe  removal  capabilities  of  the 

27  treatment  system  relative  to  existing  equipment.  Comparisons  shall  be  provided  in  such 

28  areas  as:  treatment  capabilities,  requirements  for  chemicals  and  materials,  power,  labor 

29  requirements,  suitability  for  process  monitoring  and  operation  from  remote  locations, 

30  ability  to  be  managed  by  part-time  operators; 

31  •  Discussion  of  the  known  limitations  of  the  equipment.  The  following  operational  details 

32  shall  be  included:  the  range  of  feed  water  quality  suitable  for  treatment  with  the 

33  equipment,  the  upper  limits  for  concentrations  of  regulated  contaminants  that  can  be 

34  removed  to  concentrations  below  the  MCL,  level  of  operator  skill  required  to  successfully 
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1  use  the  equipment. 

2 

3 

4  4.0  EXPERIMENTAL  DESIGN 

5 

6  This  section  discusses  the  objectives  of  the  verification  testing,  factors  that  must  be  considered  to 

7  meet  the  performance  objectives,  and  the  statistical  analysis  and  other  means  tiiat  the  NSF  will  use 

8  to  evaluate  the  results  of  the  verification  testing. 

9 

10  4.1  Objectives 

11 

12  The  objectives  of  this  verification  testing  are  to  evaluate  equipment  in  the  following  areas:  1) 

13  performance  relative  to  the  Manufacturer’s  stated  range  of  equipment  capabilities;  2)  performance 

14  relative  to  the  DBP  precursor  removal  requirements  of  enhanced  coagulation  as  part  of  the  proposed 

15  D/DBP  Rule  and  any  other  specific  or  anticipated  water  quality  regulation  (i.e..  Enhanced  Surface 

16  Water  Treatment  Rule);  3)  Ae  impacts  of  variations  in  feed  water  quality  fsuch  as  TOC.  DOC, 

17  temperature,  turbidity,  particle  concentration,  microbial  concentration.  pH.  alkalinity,  etc.)  on 

18  equipment  performance;  4)  the  logistical,  human,  and  economic  resources  necessary  to  operate  the 

19  equipment;  and  5)  the  reliability,  ruggedness,  cost,  range  of  usefulness,  and  ease  of  operation. 

20 

21  The  Manufacturer  shall  be  responsible  for  selection  of  those  treatment  challenges  listed  in  NSF  test 

22  plans  that  are  most  appropriate  for  their  equipment.  For  example,  if  equipment  is  only  intended  for 

23  removal  of  DBP  precursors,  there  would  be  no  need  to  conduct  testing  to  evaluate  the  removal  of 

24  hardness  ions  or  metal  ion  species.  However,  it  should  be  noted  that  many  of  the  packaged  and/or 

25  modular  drinking  water  treatment  systems  participating  in  the  DBP  Precursor  Removal  Verification 

26  Testing  Program  will  be  capable  of  achieving  multiple  water  treatment  objectives.  Although  this 

27  protocol  for  DBP  precursor  removal  and  the  associated  Verification  Testing  Plans  are  oriented 

28  towards  removal  of  DBP  precursors,  the  Manufacturer  may  want  to  look  at  the  treatment  system’s 

29  removal  capabilities  for  additional  water  quality  parameters. 

30 

31  4.2  Equipment  Characteristics 

32 

33  This  section  discusses  factors  that  will  be  considered  in  the  design  and  implementation  of  the 

34  Equipment  Verification  Testing  Program.  These  factors  include  ease  of  operation,  degree  of  operator 

35  attention  required,  response  of  equipment  and  treatment  process  to  ehanges  in  feed  water  quality, 

36  electrical  requirements,  system  reliability  features  including  redundancy  of  components,  feed  flow 

37  requirements,  discharge  requirements,  spatial  requirements  of  the  equipment  (footprint),  imit 

38  processes  included  in  treatment  train  and  chemicals  needed. 

39 

4  0  Verification  testing  procedures  shall  simulate  routine  conditions  as  much  as  possible  and  in  most  cases 
41  testing  may  be  done  in  the  field.  Under  such  circumstances,  simulation  of  field  conditions  would  not 
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1  be  necessary. 

2 

3  4.2.1  Qualitative  Factors 

4 

5  Some  factors,  while  important,  are  difficult  or  impossible  to  quantify.  These  are  considered 

6  qualitative  factors.  Important  factors  that  cannot  easily  be  quantified  are  the  modular  nature  of  the 

7  equipment,  the  safety  of  the  equipment,  the  portability  of  equipment,  and  the  logistical  requirements 

8  necessary  for  using  it. 

9 

10  Typical  qualitative  factors  to  be  discussed  are  listed  below,  and  others  may  be  added.  The 

11  Manufacturer  FOD  shall  discuss  those  factors  that  are  appropriate  to  the  test  equipment. 

12 

13  •  Reliability  or  susceptibility  to  environmental  conditions 

14  •  Equipment  safety 

15  •  Effect  of  operator  experience  on  results. 

16 

17  4.2.2  Quantitative  Factors 

18 

1 9  Many  factors  of  the  equipment  characteristics  can  be  quantified  by  various  means  in  this  Verification 

20  Testing  Program.  Some  can  be  measured  while  others  cannot  be  controlled.  Typical  quantitative 

21  factors  to  be  discussed  are  listed  below,  and  others  may  be  added.  The  Manufacturer  FOD  shall 

22  discuss  those  factors  that  are  appropriate  to  the  test  equipment. 

23 

24  •  Power  and  consumable  supply  (such  as  chemical  and  materials)  requirements 

25  •  Cost  of  operation,  expendables,  and  waste  disposal 

26  •  Hydrodynamics  of  packaged  plant  system 

27  •  Length  of  operating  cycle. 

28 

29  These  quantitative  factors  will  be  used  as  an  initial  benchmark  to  assess  equipment  performance. 

30 

31  4.3  Water  Quality  Considerations 

32 

33  The  primary  treatment  goal  of  the  equipment  employed  in  this  Verification  Testing  Program  is  to 

34  achieve  removal  of  DBP  precursors  found  in  feed  waters  (or  raw  waters)  such  that  product  waters 

35  are  of  acceptable  water  quality  (with  limited  presence  of  allogenic  contaminants).  The  driving  force 

36  for  the  goal  of  precursor  removal  is  to  achieve  compliance  with  the  proposed 

37  Disinfectant/Disinfection  By-Product  (D/DBP)  Rule  and  the  proposed  Groundwater  Disinfection  Rule 

38  under  the  Safe  Drinking  Water  Act.  The  experimental  design  in  the  Manufacturer  FODs  shall  be 
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1  developed  so  the  relevant  questions  about  water  treatment  equipment  capabilities  can  be  answered. 

2 

3  Manufacturers  should  carefully  consider  the  capabilities  and  limitations  of  their  equipment  and 

4  prepare  Manufacturer  FODs  that  sufficiently  challenge  their  equipment.  The  Manufacturer  should 

5  adopt  an  experimental  approach  to  verification  testing  that  would  provide  a  broad  market  for  their 

6  products,  while  recognizing  the  limitations  of  the  equipment,  and  not  conducting  precursor  removal 

7  testing  that  would  be  beyond  the  capabilities  of  the  equipment.  A  wide  range  of  contaminants  or 

8  water  quality  problems  that  can  be  addressed  by  water  treatment  equipment  varies,  and  some 

9  packaged  treatment  equipment  can  address  a  broader  range  of  problems  than  other  types. 

10  Manufacturers  shall  use  NSF  Equipment  Verification  Testing  Plans  as  the  basis  for  the  specific 

11  Manufacturer  FODs. 

12 

13  4.3.1  Feed  Water  Quality 

14 

15  One  of  the  key  aspects  related  to  demonstration  of  equipment  performance  in  the  Verification  Testing 

16  Program  is  the  range  of  feed  water  quality  that  can  be  treated  successfully.  The  Manufacturer  should 

17  consider  the  influence  of  feed  water  quality  on  the  quality  of  treated  waters  produced  by  the 

18  packaged  plant,  such  that  product  waters  meet  the  stated  water  quality  goals  (in  terms  of  disinfection 

19  by-product  concentrations)  or  regulatory  requirements  for  precursor  removals.  As  the  range  of  feed 

20  water  quality  that  can  be  treated  by  the  equipment  becomes  broader,  the  potential  applications  for 

21  treatment  equipment  with  verified  performance  capabilities  may  also  increase.  Characteristics  of  feed 

22  water  quality  that  can  be  important  for  treatment  equipment  intended  to  remove  DBP  precursors  are: 

23 

24  •  dissolved  organic  carbon  (DOC),  total  organic  carbon  (TOC),  or  UV-254  absorbance 

25  •  biological  dissolved  organic  carbon  (BDOC)  or  assimilable  organic  carbon  (AOC) 

26  •  turbidity,  particle  concentration 

27  •  pH  and  alkalinity 

28  •  temperature,  with  temperatures  near  fireezing  having  potential  for  the  most  difficult  treatment 

29  conditions 

30  •  total  dissolved  solids  (TDS),  and  other  individual  inorganic  parameters 

31  •  presence  of  background  microbial  populations  including  algae,  bacteria,  viruses  and  protozoa 

32  and  other  organisms 

33  •  Total  Kjeldahl  Nitrogen  (TKN),  ammonia  nitrogen 

34 

35  One  of  the  questions  often  asked  by  regulatory  engineers  in  approval  of  packaged  water  treatment 

36  equipment  is:  "Has  it  been  shown  to  work  on  the  water  where  you  propose  to  put  it?"  By  covering 

37  a  large  range  of  water  qualities  the  verification  testing  is  more  likely  to  provide  an  affirmative  answer 

38  to  that  question. 
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1  4.3.2  Treated  Wafer  Quality 

2 

3  Production  of  treated  water  of  a  high  quality,  with  low  concentrations  of  precursors  to  DBPs  shall 

4  be  the  primary  goal  of  the  packaged  and/or  modular  water  treatment  systems  included  in  this 

5  Equipment  Verification  Testing  Program.  If  a  Manufacturer  states  that  water  treatment  equipment 

6  can  be  used  to  treat  water  to  meet  specified  regulatory  requirements  for  removal  of  DBP  precursors, 

7  the  verification  testing  must  provide  data  that  support  such  a  statement  of  capabilities,  as  appropriate. 

8  The  statement  of  capabilities  provided  by  the  Manufacturer  shall  be  related  to  the  enhanced 

9  coagulation  requirements  of  the  proposed  D/DBP  Rule  or  the  proposed  Stage  1  DBP  maximum 

10  contaminant  levels.  The  Manufacturer  shall  be  responsible  for  identification  of  the  specific  DBPs  that 

11  shall  be  monitored  during  the  Equipment  Verification  Testing  Program.  Water  quality  analysis  for 

12  the  specific  DBPs  identified  by  the  Manufacturer  shall  be  performed  by  an  NSF-qualified  laboratory. 

13  This  issue  shall  be  discussed  further  in  Section  5.2 

14 

15  In  addition,  the  Manufacturer  may  wish  to  make  a  statement  about  performance  capabilities  of  the 

1 6  equipment  for  removal  of  other  regulated  contaminants  under  the  SD  WA  that  are  not  directly  related 

17  to  DBP  precursor  removal.  For  example,  some  water  treatment  equipment  can  be  used  to  meet 

18  aesthetie  goals  that  are  not  included  as  regulatory  requirements  of  the  SDWA.  Removal  goals  for 

19  some  of  these  parameters  may  also  be  presented  in  the  Manufacturer’s  statement  of  capabilities.  A 

20  number  of  water  quality  parameters  that  may  be  useful  for  assessing  equipment  performance  of 

21  packaged  and/or  modular  treatment  systems  are  listed  below. 

22 

23  •  particle  count  or  concentration 

24  •  biological  dissolved  organic  carbon  (BDOC)  or  assimilable  organic  carbon  (AOC) 

25  •  heterotrophic  plate  count  bacteria  (HPC) 

26  •  color,  taste  and  odor 

27  •  total  dissolved  solids 

28  •  hardness  ions 

29  •  iron  and  manganese 

30 

31  4.4  Disinfection  By-Product  Formation  Testing 

32 

33  For  evaluation  of  the  DBP  precursor  concentrations,  the  standardized  ICR  approach  of  the  Uniform 

34  Formation  Conditions  (UFC)  shall  be  employed  in  this  Verification  Testing  Program.  Selected 

35  samples  shall  be  prepared  for  THM  and  HAA  analysis  using  the  following  procedure  which  will 

36  provide  the  standardized  set  of  representative  chlorination  conditions: 

37 

38  •  Incubation  time:  24  +/- 1  hours 
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1  •  Incubation  temperature:  20.0+/- 1.0  °C 

2  •  Buffered  pH:  8.0  +/-  0.2 

3  •  24-hour  Chlorine  Residual:  1 .0  +/-  0.4  mg  Clj/L 

4 

5  For  these  conditions,  the  chlorine  dose  required  to  achieve  the  target  chlorine  residual  can  be 

6  determined  by  first  conducting  a  demand  study  with  the  water  sample.  Since  the  TOC  and  DOC 

7  concentrations  of  a  water  can  vary  over  the  course  of  a  test  run,  Ihe  chlorine  demand  of  a  given  water 

8  may  also  vary.  The  chlorine  dose  must  therefore  be  varied  according  to  the  chlorine  demand  of  the 

9  water.  Frequency  of  sampling  and  SDSDBP  analysis  shall  be  specified  by  the  individual  test  plans 

10  used  for  the  Equipment  Verification  Testing  Program  and  shall  also  be  stipulated  in  the  Manufacturer 

11  FOD. 

12 

13  4.5  Recording  data 

14 

15  For  all  DBP  precursor  experiments,  data  should  be  maintained  on  the  pH,  temperature  and  other 

16  water  quality  parameters  listed  in  Sections  4.3.1  and  4.3.2  above.  The  following  items  of  information 

17  shall  also  be  maintained  for  each  experiment: 

18 

19  •  Type  of  chemical  addition,  dose  and  chemical  combination,  where  applicable  (e.g.,  alum, 

20  cationic  polymer,  cuiionic  polymer,  ozone,  monochloramine,  scale  inhibitor,  etc.); 

21  •  Water  type  (raw  water,  pretreated  feed  water,  product  water,  waste  water); 

22  •  Experimental  run  (e.g.  P‘  run,  run,  3^'*  run,  etc.); 


23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 


4.6  Recording  Statistical  Uncertainty 

For  the  analytical  data  obtained  during  verification  testing,  95%  confidence  intervals  shall  be 
calculated  by  &e  field  testing  organization  for  selected  water  quality  parameters.  The  specific  testing 
plans  shall  specify  which  water  quality  parameters  shall  be  subjected  to  the  requirements  of 
confidence  interval  ealculation.  As  the  name  implies,  a  confidence  interval  describes  a  population 
range  in  which  any  individual  population  measurement  may  exist  with  a  specified  percent  confidence. 
The  following  formula  shall  be  employed  for  confidence  interval  calculation: 

r  5  ^ 

Confidence  Interval  —X±t  „  — j= 

where:  X  is  the  sample  mean; 

S  is  the  sample  standard  deviation; 

n  is  the  number  of  independent  measurements  included  in  the  data  set;  and 
t  is  the  Student’s  t  distribution  value  with  n-1  degrees  of  freedom; 
a  is  the  significance  level,  defined  for  95%  confidence  as:  1  -  0.95  =  0.05. 
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1  According  to  the  95%  confidence  interval  approach,  the  cc  term  is  defined  to  have  the  value  of  0.05, 

2  thus  simplifying  the  equation  for  the  95%  confidence  interval  in  the  following  manner: 

S 

95%  confidence  interval  X  t  ,0975 

3  ’■ 

4  With  input  of  the  analytical  results  for  pertinent  water  quality  parameters  into  the  95%  confidence 

5  interval  equation,  the  output  will  appear  as  the  sample  mean  value  plus  or  minus  the  second  term. 

6  The  results  of  this  statistical  calculation  may  also  be  presented  as  a  range  of  values  falling  within  the 

7  95%  confidence  interval.  For  example,  file  results  of  the  confidence  interval  calculation  may  provide 

8  the  following  information:  520  +/-  38.4  mg/L,  with  a  95%  confidence  interval  range  described  as 

9  (481.6,558.4). 

10 

11  Calculation  of  confidence  intervals  shall  not  be  required  for  equipment  performance  results  (e.g.,  filter 

12  run  length,  cleaning  efficiency,  in-line  turbidity  or  in-line  particle  counts,  etc.)  obtained  during  the 

13  equipment  testing  verification  program.  However,  as  specified  by  the  Manufacturer,  calculation  of 

14  confidence  intervals  may  be  required  for  such  analytical  parameters  as  TOC,  DOC,  grab  samples  of 

15  turbidity,  THMs,  HAAs.  In  order  to  provide  sufficient  anal5dical  data  for  statistical  analysis,  the  Field 

16  Testing  Organization  shall  collect  three  discrete  water  samples  at  one  set  of  operational  conditions 

17  for  each  of  the  specified  water  quality  parameters  during  a  designated  testing  period.  The  procedures 

18  and  sampling  requirements  shall  be  provided  in  detail  in  the  Verification  Testing  Plan. 

19 

20  4.7  Verification  Testing  Schedule 

21 

22  Verification  testing  activities  include  equipment  set-up,  initial  operation,  verification  operation,  and 

23  sampling  and  analysis.  Initial  operations  are  intended  to  be  conducted  so  that  Manufacturers  can  test 

24  their  equipment  and  be  sure  it  is  functioning  as  intended.  If  feed  water  (or  source  water)  quality 

25  influences  operation  and  performance  of  equipment  being  tested,  the  initial  operations  period  serves 

26  as  the  shake-down  period  for  determining  appropriate  operating  parameters. 

27 

28  For  water  treatment  equipment  involving  removal  of  DBP  precursors,  an  initial  period  of  bench-scale 

29  testing  of  feed  water  followed  by  treatment  equipment  operation  may  be  needed  to  determine  the 

30  appropriate  operational  parameters  for  testing  equipment.  A  number  of  operational  may  require 

31  adjustment  to  achieve  successful  functioning  of  the  process  train;  these  parameters  may  include  but 

32  are  not  limited  to:  process  rates,  feed  water  pH,  chemical  dosages,  chemical  types  where  appropriate 

33  and  equipment  operations  procedures  that  will  result  in  successful  functioning  of  the  process  train. 

34 

35  The  timing  for  verification  testing  shall  be  designated  on  the  basis  of  four  annual  testing  episodes  in 

36  order  to  cover  a  range  of  water  quality  conditions  experienced  in  an  annual  period.  For  example. 

37  climatic  changes  between  rainv  and  drv  seasons  mav  produce  substantial  variability  in  feed  watff 

38  turbidity.  Cold  weather  operations  will  be  an  important  component  of  seasonal  water  quality  testing 

39  because  of  the  impact  of  cold  temperatures  (1  “C  to  5  ^  C)  on  water  viscosity,  diffusional  processes 

40  and  characteristics  of  raw  water  DBP  precursor  materials. 
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1 

2  Content  of  Manufacturer  Field  Operations  Document  Regarding  Experimental  Design: 

3 

4  The  Manufacturer  shall  be  responsible  for  including  the  following  elements  in  the  Manufactuwe 

5  Field  Operations  Document: 

6 

7  •  Identification  of  the  qualitative  and  quantitative  factors  of  equipment  operation  to  be 

8  addressed  in  the  Verification  Testing  Program. 

9  •  Identification  and  discussion  of  the  particular  water  treatment  issues  and  dissolved 

10  organic  carbon  concentrations  that  the  equipment  is  designed  to  address,  how  the 

11  equipment  will  solve  the  problem,  and  who  would  be  the  potential  users  of  the  equipment. 

12  •  Identification  of  the  range  of  key  water  quality  parameters,  given  in  applicable  NSF 

13  Testing  Plans,  which  the  equipment  is  intended  to  address  and  for  which  the  equipment  is 

14  applicable. 

15  •  Identification  of  the  key  parameters  of  treated  water  quality  and  analytical  methods  that 

16  will  be  used  for  evaluation  of  equipment  performance  during  the  removal  of  DBP 

17  precursors.  Parameters  of  significance  for  treated  water  quality  were  listed  above  in 

18  Sections  4.3.2  and  4.3.2.  and  in  applicable  NSF  Testing  Plans. 

19  •  Description  of  data  recording  protocol  for  equipment  operation,  feed  water  quality 

20  parameters,  and  treated  water  quality  parameters. 

21  •  Description  of  the  confidence  interval  calculation  procedure  for  selected  water  quality 

22  parameters. 

23  •  Detailed  outline  of  the  verification  testing  schedule,  with  reyard  to  annual  testing  periods 

24  that  will  cover  an  appropriate  range  of  annual  climatic  conditions,  (i.e..  different 

25  temperature  conditions,  seasonal  differences  between  rainv  and  dry  conditions). 
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1  5.0  FIELD  OPERATIONS  PROCEDURES 

2 

3  5.1  Equipment  Operations  and  Design 

4 

5  The  NSF  Verification  Testing  Plan  specifies  procedures  that  shall  be  used  to  ensure  the  accurate 

6  documentation  of  both  equipment  performance  and  treated  water  quality.  Careful  adherence  to 

7  these  procedures  will  result  in  definition  of  verifiable  performance  of  equipment.  (Note  that  this 

8  protocol  may  be  associated  with  a  number  of  different  NSF  Equipment  Verification  Testing  Plans 

9  for  different  types  of  process  equipment  capable  of  achieving  removal  of  DBP  precursors). 

10 

11  Design  aspects  of  water  treatment  process  equipment  often  provide  a  basis  for  approval  by  state 

12  regulatory  engineers  and  can  be  used  to  ascertain  if  process  equipment  intended  for  larger  or 

13  smaller  flows  than  that  evaluated  in  the  Verification  Testing  Program  actually  involves  the  same 

14  operating  parameters  that  were  relevant  to  the  verification  testing.  Specific  design  aspects  to  be 

15  included  in  the  Manufacturer  FOD  are  provided  in  detail,  in  the  Manufacturer  Responsibilities 

16  section  below. 

17 

18  Initial  operations  of  the  precursor  removal  equipment  will  allow  equipment  Manufacturers  to 

19  refine  their  operating  procedures  and  to  make  operational  adjustments  as  needed  to  successfully 

20  treat  the  feed  water.  Information  generated  through  this  period  of  operation  may  be  used  to 

21  revise  the  Manufacturer  FOD,  if  necessary.  A  failure  at  this  point  in  the  verification  testing  could 

22  indicate  a  lack  of  capability  of  the  process  equipment  and  the  verification  testing  might  be 

23  canceled. 

24 

25  5.2  Selection  of  Analytical  Laboratory  and  Field  Testing  Organization 

26 

27  To  assess  the  performance  of  the  equipment,  the  quality  of  the  treated  water  produced  using  the 

28  equipment  shall  be  determined  by  analysis  at  an  NSF-qualified  analytical  laboratory  with  proven 

29  experience  in  detection  and  measurement  of  total  organic  carbon,  dissolved  organic  carbon, 

30  trihalomethanes,  haloacetic  acids,  bromate,  chlorate  and  other  regulated  DBFs.  In  all  cases, 

31  current  APHA  Standard  Methods  procedures  shall  be  used  in  analysis  of  specified  water  quality 

32  parameters.  The  NSF  may  provide  a  list  of  qualified  laboratories  from  which  Manufacturers  can 

33  select  for  submission  of  samples  for  water  quality  analysis.  Because  of  the  variability  of 

34  acceptance  of  laboratories  from  state  to  state,  use  of  analytical  laboratories  certified  in  a  large 

35  number  of  states  is  recommended.  Furthermore,  the  selected  analytical  laboratory  must  be 

36  certified  bv  the  State  in  which  the  verification  testing  is  being  performed.  Laboratories  approved 
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1  for  sample  analysis  for  the  EPA's  Information  Collection  Rule  would  have  nationally  recognized 

2  capabilities.  Analytical  results  from  the  laboratory  are  to  be  provided  directly  to  the  NSF  to 

3  maintain  data  integrity. 

4 

5  For  field  testing  operations,  the  Manufacturer  shall  employ  an  NSF-qualified  Field  Testing 

6  Organization;  the  list  of  qualified  organizations  may  include  engineering  consulting  firms, 

7  universities,  or  other  qualified  scientific  organizations  with  experience  operating  pilot  plant 

8  equipment. 

9 

10  5.3  Communications,  Documentation,  Logistics,  and  Equipment 

11 

12  NSF  shall  communicate  regularly  with  the  verification  testing  participants  to  coordinate  all  field 

13  activities  associated  with  this  verification  testing  and  to  resolve  any  logistical,  technical,  or  QA 

14  issues  that  may  arise  as  the  verification  testing  progresses.  The  successful  implementation  of  the 

15  verification  testing  will  require  detailed  coordination  and  constant  communication  between  all 

16  verification  testing  participants. 

17 

18  All  Manufacturer/NSF  field  activities  shall  be  thoroughly  documented.  Field  documentation  will 

19  include  field  logbooks,  photographs,  field  data  sheets,  and  chain-of-custody  forms.  The  qualified 

20  testing  organization  shall  be  responsible  for  maintaining  all  field  documentation.  Field  notes  shall 

21  be  kept  in  a  bound  logbook.  Each  page  shall  be  sequentially  numbered  and  labeled  with  the 

22  project  name  and  number.  Field  logbooks  shall  be  used  to  record  all  water  treatment  equipment 

23  operating  data.  Completed  pages  shall  be  signed  and  dated  by  the  individual  responsible  for  the 

24  entries.  Errors  shall  have  one  line  drawn  through  them  and  this  line  shall  be  initialed  and  dated. 

25 

26  All  photographs  shall  be  logged  in  the  field  logbook.  These  entries  shall  include  the  time,  date, 

27  direction,  subject  of  the  photograph,  and  the  identity  of  the  photographer.  Any  deviations  from 

28  the  approved  final  Manufacturer  FOD  shall  be  thoroughly  documented  in  the  field  logbook  and 

2  9  provided  to  the  NSF . 

30 

31  Original  field  sheets  and  chain-of-custody  forms  shall  accompany  all  samples  shipped  to  the 

32  analytical  laboratory.  Copies  of  field  sheets  and  chain-of-custody  forms  for  all  samples  shall  be 

3  3  provided  to  the  NSF . 

34 

35  5.4  Equipment  Operation  and  Water  Quality  Sampling  for  Verification  Testing 
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1 

2  The  qualified  testing  organization  will  supervise  equipment  operation  and  water  quality  sampling 

3  and  analysis  during  the  verification  phase  of  testing,  using  the  procedures  described  below  and  in 

4  the  specific  Verification  Testing  Plans.  The  NSF  will  oversee  or  audit  these  activities.  All  field 

5  activities  shall  conform  with  requirements  provided  in  the  Manufacturer  FOD  that  was  developed 

6  and  NSF-approved  for  the  verification  testing  being  conducted. 

7 

8  If  unanticipated  or  unusual  situations  are  encountered  that  may  alter  the  plans  for  equipment 

9  operation,  water  quality  sampling,  or  data  quality,  the  situation  must  be  discussed  with  the  NSF 

10  technical  lead.  Any  deviations  from  the  approved  final  Manufacturer  FOD  shall  be  thoroughly 

11  documented. 

12 

13  During  routine  operation  of  water  treatment  equipment,  the  total  number  of  hours  during  which 

14  the  equipment  is  operated  each  day  shall  be  documented.  In  addition,  the  number  of  hours  each 

15  day  during  which  the  operator  was  working  at  the  treatment  plant  performing  tasks  related  to 

16  water  treatment  and  the  operation  of  the  treatment  equipment  shall  be  documented.  Furthermore, 

17  the  tasks  performed  during  equipment  operation  shall  be  described  by  the  qualified  Testing 

18  Organization,  the  Water  System  or  the  Plant  Operator. 

19 

20  Content  of  Manufacturer  Field  Operations  Document  Regarding  Field  Operations 

21  Procedures: 

22 

23  The  Manufacturer  shall  be  responsible  for  including  the  following  elements  in  the  Manufacturer 

24  Field  Operations  Document: 

25 

26  •  A  table  summary  of  the  proposed  time  schedule  for  operating  and  testing, 

27  •  Field  operating  procedures  for  the  equipment  and  performance  testing,  based  upon  the 

28  NSF  Equipment  Verification  Testing  Plan  with  listing  of  operating  parameters,  ranges 

29  for  feed  water  quality,  and  the  sampling  and  analysis  strategy. 

30 

31  Manufacturer  Responsibilities: 

32 

33  •  Provision  of  all  equipment  needed  for  field  work  associated  with  this  verification  testing; 

34  •  Provision  of  a  complete  list  of  all  equipment  to  be  used  in  the  verification  testing.  A  table 

35  format  is  suggested; 


This  Draft  Protocol  shall  not  be  duplicated  in  whole  or  in  part  without  the  written  permission  of  NSF  International 


Page  21 


1  •  Provision  of  field  operating  procedures. 

2 

3 

4  6.0  QUALITY  ASSURANCE  PROJECT  PLAN  (QAPP) 

5 

6  The  QAPP  for  this  verification  testing  specifies  procedures  that  shall  be  used  to  ensure  data  quality 

7  and  integrity.  Careful  adherence  to  these  procedures  will  ensure  that  data  generated  from  the 

8  verification  testing  will  provide  sound  analytical  results  that  can  serve  as  the  basis  for  performance 

9  verification. 

10 

11  6.1  Purpose  and  Scope 

12 

13  The  primary  purpose  of  this  section  is  to  outline  steps  that  shall  be  taken  by  operators  of  the 

14  equipment  and  by  the  analytical  laboratory  to  ensure  that  data  resulting  from  this  verification  testing 

15  is  of  known  quality  and  that  a  sufficient  number  of  critical  measurements  are  taken. 

16 

17  6.2  Quality  Assurance  Responsibilities 

18 

19  The  Manufacturer  project  manager  is  responsible  for  coordinating  the  preparation  of  the  QAPP  for 

20  this  verification  testing  and  for  its  approval  by  the  NSF.  The  qualified  testing  organization  project 

2 1  manager,  with  oversight  from  NSF,  shall  ensure  that  the  QAPP  is  implemented  during  all  verification 

22  testing  activities. 

23 

24  The  entire  Manufacturer  FOD  including  the  QAPP  must  be  approved  by  the  Manufacturer  and  the 

25  NSF  before  the  verification  testing  can  proceed.  The  NSF  must  review  and  either  approve  the  QAPP 

26  or  provide  reasons  for  rejection  of  the  QAPP  along  with  suggestions  on  how  to  modify  the  QAPP 

27  to  make  it  acceptable,  provided  that  the  Manufacturer  has  made  a  good  faith  effort  to  develop  an 

28  acceptable  QAPP  (i.e.  the  QAPP  is  75  to  80%  acceptable  with  only  minor  changes  needed  to  produce 

29  an  acceptable  plan.  NSF  will  not  write  QAPPs  for  Manufacturers.) 

30 

31  A  number  of  individuals  may  be  responsible  for  monitoring  equipment  operating  parameters  and  for 

32  sampling  and  analysis  QA/QC  throughout  the  verification  testing.  Primary  responsibility  for  ensuring 

33  that  both  equipment  operation  and  sampling  and  analysis  activities  comply  with  the  QA/QC 

34  requirements  of  the  Manufacturer  FOD  (Section  6)  shall  rest  with  the  qualified  testing  organization, 

35  with  oversight  by  the  NSF.  QA/QC  activities  for  the  equipment  shall  include  those  activities 

36  recommended  by  Manufacturer  and  those  required  by  the  NSF  to  assure  the  verification  testing  will 

37  provide  data  of  the  necessary  quality. 

38 

39  QA/QC  activities  for  the  NSF-certified  analytical  laboratory  that  analyzes  samples  sent  off-site  shall 

40  be  the  responsibility  of  that  analytical  laboratory's  supervisor.  If  problems  arise  or  any  data  appear 

4 1  unusual,  they  shall  be  thoroughly  documented  and  corrective  actions  shall  be  implemented  as  specified 
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1  in  this  section.  The  QA/QC  measurements  made  by  the  off-site  analytical  laboratory  are  dependent 

2  on  the  analytical  methods  being  used. 

3 

4  6.3  Data  Quality  Indicators 

5 

6  The  data  obtained  during  the  verification  testing  must  be  of  sound  quality  for  conclusions  to  be  drawn 

7  on  the  equipment.  For  all  measurement  and  monitoring  activities  conducted  for  equipment 

8  verification,  the  NSF  and  EPA  require  that  data  quality  parameters  be  established  based  on  the 

9  proposed  end  uses  of  the  data.  Data  quality  parameters  include  four  indicators  of  data  quality: 

10  representativeness,  completeness,  accuracy,  and  precision. 

11 

12  Treatment  results  generated  by  the  equipment  and  by  the  laboratory  analyses  must  be  verifiable  for 

13  the  purposes  of  this  program  to  be  fulfilled.  High  quality,  well  documented  analytical  laboratoiy 

14  results  are  essential  for  meeting  the  pxirpose  and  objectives  of  this  verification  testing.  Therefore,  the 

15  following  indicators  of  data  quality  shall  be  closely  evaluated  to  determine  the  performance  of  the 

16  equipment  when  measured  against  data  generated  by  the  analytical  laboratory. 

17 

18  6.3.1  Representativeness 

19 

20  Representativeness  refers  to  the  degree  to  which  the  data  accurately  and  precisely  represent  the 

21  conditions  or  characteristics  of  the  parameter  represented  by  the  data.  In  this  verification  testing, 

22  representativeness  will  be  ensured  by  maintaining  consistent  sample  collection  procedures,  including 

23  sample  locations,  timing  of  sample  collection,  sampling  procedures,  sample  preservation,  sample 

24  packaging,  sample  shipping,  and  sample  equipment  decontamination  (Section  5),  and  by  executing 

25  random  DBF  spiking  procedures.  Representativeness  also  will  be  ensured  by  using  each  method  at 

26  its  optimum  capability  to  provide  results  that  represent  the  most  accurate  and  precise  measurement 

27  it  is  capable  of  achieving.  For  equipment  operating  data,  representativeness  entails  collecting  a 

28  sufficient  quantity  of  data  during  operation  to  be  able  to  detect  a  change  in  operations. 

29 

30  6.3.2  Completeness 

31 

32  Completeness  refers  to  the  amount  of  data  collected  from  a  measurement  process  compared  to  the 

33  amount  that  was  expected  to  be  obtained.  For  this  verification  testing,  completeness  refers  to  the 

34  proportion  of  valid,  acceptable  data  generated  using  each  method.  The  completeness  objective  for 

35  data  generated  during  this  verification  testing  is  85  percent. 

36 

37  6.3.3  Accuracy 

38 

39  For  water  quality  analyses,  accuracy  refers  to  the  difference  between  a  sample  result  and  the  reference 

40  or  true  value  for  the  sample.  Loss  of  accuracy  can  be  caused  hy  such  processes  as  errors  in  standards 

4 1  preparation,  equipment  calibrations,  loss  of  target  analyte  in  the  extraction  process,  interferences,  and 
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1  systematic  or  carryover  contamination  from  one  sample  to  the  next. 

2 

3  For  equipment  operating  parameters,  accuracy  refers  to  the  difference  between  the  reported  operating 

4  condition  and  the  actual  operating  condition.  For  water  flow,  accuracy  may  be  the  difference 

5  between  the  reported  flow  indicated  by  a  flow  meter  and  the  flow  as  actually  measured  on  the  basis 

6  of  known  volumes  of  water  and  carefully  defined  times  (bucket  and  stopwatch  technique)  as  practiced 

7  in  hydraulics  laboratories  or  water  meter  calibration  shops.  For  mixing  equipment,  accuracy  is  the 

8  difference  between  an  electronic  readout  for  equipment  RPMs  and  the  actual  measurement  based  on 

9  counted  revolutions  and  measured  time.  Accuracy  of  head  loss  measurement  can  be  determined  by 

10  using  measuring  tapes  to  check  the  calibration  of  piezometers  for  gravity  filters  or  by  checking  the 

11  calibration  of  pressure  gauges  for  pressure  filters.  Meters  and  gauges  must  be  checked  periodically 

12  for  accuracy,  and  when  proven  to  be  dependable  over  time,  the  time  interval  between  accuracy  checks 

13  can  be  increased. 

14 

15  6.3.4  Precision 

16 

17  Precision  refers  to  the  degree  of  mutual  agreement  among  individual  measurements  and  provides  an 

18  estimate  of  random  error. 

19 

20  6.3.5  Statistical  Uncertainty 

21 

22  Statistical  uncertainty  of  the  water  quality  parameters  analyzed  shall  be  evaluated  through  calculation 

23  of  the  95%  confidence  interval  around  the  sample  mean.  Description  of  the  confidence  interval 

24  calculation  is  provided  in  Section  4.6  -  Recording  Statistical  Uncertainty. 

25 

26  6.4  Water  Quality  and  Operational  Control  Checks 

27 

28  This  section  describes  the  QC  requirements  that  apply  to  both  the  treatment  equipment  and  the  on- 

29  site  measurement  of  water  quality  parameters.  It  also  contains  a  discussion  of  the  corrective  action 

30  to  be  taken  if  the  QC  parameters  fall  outside  of  the  evaluation  criteria. 

31 

32  The  quality  control  checks  provide  a  means  of  measuring  the  quality  of  data  produced.  The 

33  Manufacturer  may  not  need  to  use  all  the  ones  identified  in  this  section.  The  selection  of  the 

34  appropriate  quality  control  checks  depends  on  the  equipment,  the  experimental  design  and  the 

35  performance  goals.  The  selection  of  quality  control  checks  will  be  based  on  discussions  among  the 

36  Manufacturer  and  the  NSF.  Some  types  of  quality  control  checks  applicable  to  operating  water 

37  treatment  equipment  were  described  in  Section  6.3.4. 

38 

39  6.4.1  Quality  Control  for  Equipment  Operation 

40 

41  This  section  will  explain  the  methods  to  be  used  to  check  on  the  accuracy  of  equipment  operating 
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1  parameters  and  the  frequency  with  which  these  quality  control  checks  will  be  made.  A  key  aspect  of 

2  the  Equipment  Verification  Testing  Program  is  to  provide  operating  results  that  will  be  widely 

3  accepted  by  state  regulatory  engineers.  If  the  quality  of  the  equipment  operating  data  can  not  be 

4  verified,  then  the  water  quality  analytical  results  may  be  of  no  value.  Because  water  can  not  be 

5  treated  if  equipment  is  not  operating,  obtaining  valid  equipment  operating  data  is  a  prime  concern  for 

6  verification  testing. 

7 

8  An  example  of  the  need  for  QC  for  equipment  operations  is  an  incident  of  state  rejection  of  test  data 

9  because  the  treatment  equipment  had  no  flow  meter  to  use  for  determining  engineering  and  operating 

10  parameters  related  to  flow. 

11 

12  6.4.2  Water  Quality  Data 

13 

14  After  treatment  equipment  is  being  operated  and  water  is  being  treated,  the  results  of  the  treatment 

15  are  interpreted  in  terms  of  water  quality.  Therefore  the  quality  of  water  sample  analytical  results  is 

16  just  as  important  as  the  quality  of  the  equipment  operating  data.  Most  QA  plans  emphasize  analytical 

17  QA.  The  important  aspects  of  sampling  and  analytical  QA  are  given  below: 

18 

19  6.4.2.1  Triplicate  Analysis  of  Selected  Water  Quality  Parameters 

20 

21  Triplicate  samples  shall  be  analyzed  for  selected  water  quality  parameters  at  specified  intervals  in 

22  order  to  determine  the  precision  of  analysis.  The  procedure  for  determining  samples  to  be  analyzed 

23  in  triplicate  shall  be  provided  in  each  Verification  Testing  Plan  with  the  required  frequency  of  analysis 

24  and  Ae  approximate  number.  The  triplicate  analysis  shall  be  performed  according  to  the  requirements 

25  for  calculation  of  95%  confidence  intervals,  as  presented  in  Section  4.6. 

26 

27  6.4.2.2  Method  Blanks 

28 

2  9  Method  blanks  are  used  for  selected  water  quality  parameters  to  evaluate  analytical  method-induced 

30  contamination,  which  may  cause  false  positive  results. 

31 

32  6.4.2.3  Spiked  Samples 

33 

34  The  use  of  spiked  samples  will  depend  on  the  testing  program,  and  the  contaminants  to  be  removed. 

35  If  spiked  samples  are  to  be  used  specify  the  procedure,  frequency,  acceptance  criteria,  and  actions 

36  if  criteria  are  not  met. 

37 

38  6.4.2.4  Travel  Blanks 

39 

40  Travel  blanks  for  selected  water  quality  parameters  shall  be  provided  to  the  analytical  laboratory  to 

41  evaluate  travel-related  contamination. 
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Performance  Evaluation  Samples  for  On-Site  Water  Quality  Testing 


1 
2 

3  Performance  evaluation  (PE)  samples  are  samples  whose  composition  is  unknown  to  the  analyst  that 

4  are  used  to  evaluate  analytical  performance.  Analysis  of  PE  samples  shall  be  conducted  for  selected 

5  water  quality  parameters  before  pilot  testing  is  initiated  by  submission  of  samples  to  the  analytical 

6  laboratory  and  to  the  equipment  testing  organizations,  if  appropriate.  The  control  limits  for  the  PE 

7  samples  will  be  used  to  evaluate  the  equipment  testing  organization's  and  analytical  laboratory's 

8  method  performance.  One  kind  of  PE  sample  that  would  be  used  for  on-site  QA  in  most  studies  done 

9  under  this  protocol  would  be  a  turbidity  PE  sample. 

10 

11  PE  samples  come  with  statistics  about  each  sample  which  have  been  derived  from  the  analysis  of  the 

12  sample  by  a  number  of  laboratories  using  EPA-approved  methods.  These  statistics  include  a  true 

13  value  of  the  PE  sample,  a  mean  of  the  laboratory  results  obtained  from  the  analysis  of  the  PE  sample, 

14  and  an  acceptance  range  for  sample  values.  The  analytical  laboratory  is  expected  to  provide  results 

15  from  the  analysis  of  the  PE  samples  that  meet  the  performance  objectives  of  the  verification  testing. 

16 

17 

18  6.6  Data  Reduction,  Validation,  and  Reporting 

19 

20  To  maintain  good  data  quality,  specific  procedures  shall  be  followed  during  data  reduction,  validation, 

21  and  reporting.  These  procedures  are  detailed  below. 

22 

23  6.6.1  Data  Reduction 

24 

25  Data  reduction  refers  to  the  process  of  converting  the  raw  results  from  the  equipment  into 

26  concentration  or  other  data  in  a  form  to  be  used  in  the  comparison.  The  procedures  to  be  used  will 

27  be  equipment  dependent.  The  purpose  of  this  step  is  to  provide  data  which  will  be  used  to  verify  the 

28  statement  of  performance  capabilities.  These  data  shall  be  obtained  from  logbooks,  instrument 

29  outputs,  and  computer  outputs  as  appropriate. 

30 

31  6.6.2  Data  Validation 

32 

33  The  operator  shall  verify  the  completeness  of  the  appropriate  data  forms  and  the  completeness  and 

34  correctness  of  data  acquisition  and  reduction.  The  field  team  supervisor  or  another  technical  person 

35  shall  review  calculations  and  inspect  laboratory  logbooks  and  data  sheets  to  verify  accuracy, 

36  completeness.  Calibration  and  QC  data  will  be  examined  by  the  individual  operators  and  the 

37  laboratory  supervisor.  Laboratory  and  project  managers  shall  verify  that  all  instrument  systems  are 

38  in  control  and  that  QA  objectives  for  accuracy,  completeness,  and  method  detection  limits  have  been 

39  met. 

40 

4 1  Analytical  outlier  data  are  defined  as  those  QC  data  lying  outside  a  specific  QC  objective  window  for 
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1  precision  and  accuracy  for  a  given  analytical  method.  Should  QC  data  be  outside  of  control  limits, 

2  the  analytical  laboratory  or  field  team  supervisor  will  investigate  the  cause  of  the  problem.  If  the 

3  problem  involves  an  analytical  problem,  the  sample  will  be  reanalyzed.  If  the  problem  can  be 

4  attributed  to  the  sample  matrix,  the  result  will  be  flagged  with  a  data  qualifier.  This  data  qualifier  will 

5  be  included  and  explained  in  the  final  analytical  report. 

6 

7  6.6.3  Data  Reporting 

8 

9  The  Field  Testing  Organization  shall  provide  to  the  NSF  a  list  of  all  water  quality  and  equipment 

10  operation  data  to  be  reported.  At  a  minimum,  the  data  tabulation  shall  list  the  results  for  feed  water 

11  and  treated  water  quality  analyses  and  equipment  operating  data.  All  QC  information  such  as 

12  calibrations,  blanks  and  reference  samples  are  to  be  included  in  an  appendix  to  the  report  submitted 

13  to  NSF.  All  raw  analytical  data  shall  also  be  reported  in  an  appendbc.  All  data  shall  be  reported  in 

14  hardcopy  and  electronically  in  a  common  spreadsheet  or  database  format. 

15 

16  6.7  Calculation  of  Data  Quality  Indicators 

17 

18  The  equations  for  any  data  quality  indicator  calculations  employed  shall  be  provided.  These  include: 

19  precision,  relative  percent  deviation,  standard  deviation,  confidence  interval,  accuracy,  and 

20  completeness. 

21 

22  6.8  System  Audits 

23 

24  On-site  system  audits  for  sampling  activities,  field  operations,  and  laboratories  shall  be  conducted  as 

25  specified  by  the  NSF  Equipment  Verification  Testing  Plan.  These  audits  will  be  performed  by  the 

2  6  NSF  to  determine  if  the  NSF  Equipment  Verification  Testing  Plan  is  being  implemented  as  intended. 

27  At  a  minimum.  NSF  shall  conduct  one  audit  of  the  sampling  activities,  field  operations  program  and 

28  laboratories  during  the  Verification  Testing  Smdv.  The  number  of  audits  performed  during  a  study 

29  shall  be  specified  bv  the  pertinent  Equipment  Verification  Testing  Plan.  Separate  audit  reports  will 

30  be  completed  after  the  audits  and  provided  to  the  participating  parties  through  the  NSF. 

31 

32  6.9  Reports 

33 

34  6.9.1  Status  Reports 

35 

36  The  equipment  testing  organization  shall  prepare  periodic  reports  for  the  NSF  project  managers. 

37  These  reports  shall  discuss  project  progress,  problems  and  associated  corrective  actions,  and  future 

38  scheduled  activities  associated  with  the  verification  testing.  Each  report  shall  include  an  executive 

39  summary  at  the  beginning  of  the  report  to  introduce  the  salient  issues  of  the  testing  period.  When 

4  0  problems  occur,  the  Manufacturer  and  equipment  testing  organization  project  managers  shall  discuss 

4 1  them  with  the  NSF  technical  lead,  estimate  the  type  and  degree  of  impact,  and  describe  the  corrective 
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1  actions  taken  to  mitigate  the  impact  and  to  prevent  a  recurrence  of  the  problems.  The  frequency, 

2  format,  and  content  of  these  reports  shall  be  outlined  in  the  Manufacturer  FOD. 

3 

4  6.9.2  Audit  Reports 

5 

6  Any  QA  audits  or  inspections  that  take  place  in  the  field  or  at  the  analytical  laboratory  while  the 

7  verification  testing  is  being  conducted  shall  be  formally  reported  by  the  equipment  testing 

8  organizations  to  the  NSF  project  manager  who  will  forward  them  to  the  Manufacturer  and  NSF  QC 

9  Manager  for  appropriate  actions. 

10 

11  6.10  Corrective  Action 

12 

13  Each  Manufacturer  FOD  must  incorporate  a  corrective  action  plan.  This  plan  must  include  the 

14  predetermined  acceptance  limits,  the  corrective  action  to  be  initiated  whenever  such  acceptance 

15  criteria  are  not  met,  and  the  names  of  the  individuals  responsible  for  implementation. 

16 

17  Routine  corrective  action  may  result  from  common  monitoring  activities,  such  as: 

18 

19  •  Performance  evaluation  audits 

20  •  Technical  systems  audits 

21 

22  Content  of  Manufacturer  Field  Operations  Document  Regarding  Quality  Assurance  Project 

23  Plan: 

24 

25  The  Manufacturer  shall  be  responsible  for  including  the  following  elements  in  the  Manufactuwe 

26  Field  Operations  Document: 

21 

28  •  Description  of  methodology  for  measurement  of  accuracy. 

29  •  Description  of  methodology  for  measurement  of precision. 

30  •  Description  of  the  methodology  for  use  of  blanks,  the  materials  used,  the  frequency,  the 

31  criteria  for  acceptable  method  blanks  and  the  actions  if  criteria  are  not  met. 

32  •  Description  of  any  specific  procedures  appropriate  to  the  analysis  of  the  PE  samples.  It 

33  has  to  be  clear  how  these  samples  are  going  to  be  used  in  the  verification  testingJne 

34  use  of  PE  samples  is  in  the  conduct  of  a  performance  audit  (see  Section  6. 7.1). 

35  •  Outline  of  the  procedure  for  determining  samples  to  be  analyzed  in  triplicate,  the 

36  frequency  and  approximate  number. 

37  •  Description  of  the  procedures  used  to  assure  that  the  data  are  correct. 

38  •  Listing  of  equations  used  for  any  necessary  data  quality  indicator  calculations .  These 
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1  include:  precision,  relative  percent  deviation,  standard  deviation,  confidence  interval 

2  calculation;  accuracy,  and  completeness. 

3  •  Outline  of  the  frequency,  format,  and  content  of  reports  in  the  Manufacturer  Field 

4  Operations  Document. 

5  •  Development  of  a  corrective  action  plan  in  the  Manufacturer  Field  Operations 

6  Document. 

1 

8  Manufacturer  Responsibilities: 

9 

10  •  Provision  of  all  QC  information  such  as  calibrations,  blanks  and  reference  samples  in  an 

11  appendix.  All  raw  analytical  data  shall  also  be  reported  in  an  appendix. 

12  •  Provision  of  all  data  in  hardcopy  and  electronic  form  in  a  common  spreadsheet  or 

13  database  format. 

14 

15 

1 6  7.0  DATA  MANAGEMENT  AND  ANALYSIS,  AND  REPORTING 

17 

18  7.1  Data  Management  and  Analysis 

19 

20  The  Manufacturer,  the  qualified  testing  organization  and  the  NSF  each  have  distinct  responsibilities 

21  for  managing  and  analyzing  verification  testing  data.  The  equipment  testing  organization  is 

22  responsible  for  managing  eill  the  data  and  information  generated  during  the  verification  testing.  The 

23  Manufacturer  is  responsible  for  furnishing  those  records  generated  by  the  equipment  testing 

24  organization.  The  NSF  will  be  responsible  for  analysis  and  verification  of  the  data 

25 

26  A  variety  of  data  will  be  generated  during  a  verification  testing.  Each  piece  of  data  or  information 

27  identified  for  collection  in  the  NSF  Equipment  Verification  Testing  Plan  will  need  to  be  provided  to 

28  the  NSF.  The  data  management  section  of  the  Manufacturer  FOD  shall  describe  what  types  of  data 

29  and  information  needs  to  be  collected  and  managed,  and  shall  also  describe  how  the  data  will  be 

30  reported  to  the  NSF  for  evaluation. 

31 

32  Laboratory  Analyses:  The  raw  data  and  the  validated  data  must  be  provided  to  the  NSF.  These  data 

33  shall  be  provided  in  hard  copy  and  in  electronic  format.  As  with  the  data  generated  by  the  innovative 

34  equipment,  the  electronic  copy  of  the  laboratory  data  shall  be  provided  in  a  spreadsheet,  and  a  data 

35  dictionary  shall  be  provided.  In  addition  to  the  sample  results,  all  QA/QC  summary  forms  must  be 

36  provided. 

37 

38  Other  items  that  must  be  provided  include: 
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1 

2  •  field  notebooks; 

3  •  photographs,  slides  and  videotapes  (copies); 

4  •  results  from  the  use  of  other  field  analytical  mefiiods. 

5 

6  7.2  Report  of  Equipment  Testing 

7 

8  The  qualified  testing  organization  shall  prepare  a  draft  report  describing  the  verification  testing  that 

9  was  carried  out  and  the  results  of  that  testing.  This  report  shall  include  the  following  topics: 

10 

11  •  Introduction 

12  •  Executive  Summary 

13  •  Description  and  Identification  of  Product  Tested 

14  •  Procedures  and  Methods  Used  in  Testing 

15  •  Results  and  Discussion 

16  •  Conclusions  and  Recommendations 

17  •  References 

18  •  Appendices 

19  •  Manufacturer  FOD 

20  •  QA/AC  Results 

21 

22  The  NSF  will  review  the  draft  report,  the  results  of  testing,  the  QA/QC  results,  and  will  prepare  a 

23  final  report. 

24 

25 

26  Content  of  Manufacturer  Field  Operations  Document  Regarding  Data  Management  and 

27  Analysis,  and  Reporting: 

28 

29  The  Manufacturer  shall  be  responsible  for  including  the  following  elements  in  the  Manufacture 

30  Field  Operations  Document: 

31 

32  •  Description  of  what  types  of  data  and  information  needs  to  be  collected  and  managed. 

33  •  Description  of  how  the  data  will  be  reported  to  the  NSF  for  evaluation. 

34 
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1  8.0  SAFETY  MEASURES 

2 

3  The  safety  procedures  shall  address  safety  considerations,  including  the  following  as  applicable: 

4 

5  •  conformance  with  electrical  code 

6  •  biohazards 

7  •  ventilation  of  equipment  or  of  trailers  or  buildings  housing  equipment,  if  gases  generated 

8  by  the  equipment  could  present  a  safety  hazard  (one  example  is  ozone). 

9 

10  Content  of  Manufacturer  Field  Operations  Document  Regarding  Safety: 

11 

12  The  Manufacturer  Field  Operations  Document  shall  address  safety  considerations  that  or 

13  appropriate  for  the  equipment  being  tested  and  for  the  chemicals  employed  in  the  verificatm 

14  testing. 

15 
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